Papers by Akash Kumar Mohankumar
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that evaluating NLG systems using pairwise comparisons is expensive as the number of human annotations grows linearly with k. |
| Approach: | They propose a framework to efficiently identify the top-ranked system by actively choosing system pairs for comparison using dueling bandit algorithms. |
| Outcome: | The proposed framework reduces human annotations by 80% on 13 NLG evaluation datasets spanning 5 tasks . |
Improving Retrieval in Sponsored Search by Leveraging Query Context Signals (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing models for retrieving relevant bid keywords fail to capture nuanced user intent . a new approach to enhance query understanding uses contextual signals . |
| Approach: | They propose a method to augment queries with rich contextual signals from web search results and large language models stored in an online cache. |
| Outcome: | The proposed approach outperforms context-free models in retrieving relevant bid keywords for user queries. |
Towards Transparent and Explainable Attention Models (2020.acl-main)
Copied to clipboard
Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran
| Challenge: | Recent studies on interpretability of attention distributions have led to notions of faithful and plausible explanations for a model’s predictions. |
| Approach: | They propose to modify LSTM cells to ensure that the hidden representations learned at different time steps are diverse. |
| Outcome: | The proposed model can provide a faithful explanation if a higher attention weight implies a greater impact on the model’s prediction. |
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for dialog evaluation are trained using a single relevant response and multiple random negatives. |
| Approach: | They propose a dataset to test whether model-based dialog evaluation metrics can be used to train models . they propose n-gram based metrics and embedding based ones to be used for model-driven evaluation . |
| Outcome: | The proposed model outperforms existing models on a reddit dataset on relevant responses and adversarial responses. |
Let’s Ask Again: Refine Network for Automatic Question Generation (D19-1)
Copied to clipboard
Preksha Nema, Akash Kumar Mohankumar, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran
| Challenge: | Existing AQG models produce incomplete questions which look like incomplete drafts with scope for refinement. |
| Approach: | They propose a method which mimics the human process of generating questions by first creating an initial draft and then refining it. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three datasets and improves on fluency and answerability metrics. |